Covariance & Correlation

Variance & Covariance

Variance measures the variation of a single random variable:

The diagonal entries of the covariance matrix are the variances and the other entries are the covariances:

C o v (x, y) = (\begin{array}{cc} σ (x, x) & σ (x, y) \\ σ (y, x) & σ (y, y) \end{array})

where

$σ (x, x)$ and $σ (y, y)$ are simply same as $σ_{x}^{2}$ and $σ_{y}^{2}$ , respectively.
$σ (x, y) = σ (y, x)$
which derives:

C = \frac{1}{n - 1} \sum_{i = 1}^{n} (X_{i} - \bar{X}) (X_{i} - \bar{X})^{T}

Correlation

C o r r (x, y) = \frac{C o v (x, y)}{\sqrt{σ_{x}^{2} σ_{y}^{2}}} = \frac{C o v (x, y)}{σ_{x} σ_{y}}

Covariance vs. Correlation Coefficient

Both of them measure the linear relationship between two variables.

Reference

An intuitive example of covariance matrix: https://www.youtube.com/watch?v=152tSYtiQbw&ab_channel=ritvikmath
Data science example: https://datascienceplus.com/understanding-the-covariance-matrix/
Example covariance and correlation between two random variables: https://www.youtube.com/watch/KDw3hC2YNFc